1. Field of the Invention
The present invention generally relates to video coding, and more particularly to a system of rate-distortion optimized (RDO) transform and quantization.
2. Description of Related Art
The high performance of H.264 video coding is attributed to, among others, the adoption of rate-distortion optimization (RDO) framework, which attains good balance between distortion and bitrate for mode decision and rate control.
Moreover, some methods have recently adopted the rate distortion optimization framework for another important component of a video encoder—quantization. However, a common drawback of these methods is the high computational overhead due to, for example, an exhaustive search and redundant entropy coding process to determine the quantized transform coefficients with lowest rate-distortion cost, as discussed in “Rate distortion optimization for H.264 interframe coding: a general framework and algorithms,” by E.-H. Yang and X. Yu, IEEE Trans. Image Process., vol. 16, no. 7, July 2007, and “Rate distortion optimized quantization,” by M. Karczewicz, Y. Ye and I. Chong, VCEG-AH21, January 2008.
For the reason that the transform and quantization of H.264 only considers the distortion between original and reconstructed video as the cost of compression, and conventional methods perform distortion optimization with high computational overhead, a need has thus arisen to develop a novel scheme with high efficiency and low computational complexity for a video coding process.